Entropy production in nonequilibrium steady states: A different approach and an 

exactly solvable canonical model 
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We discuss entropy production in nonequilibrium steady states by focusing on paths obtained 
by sampling at regular (small) intervals, instead of sampling on each change of the system's state. 
This allows us to study directly entropy production in systems with microscopic irreversibility, for 
the first time. The two sampling methods are equivalent, otherwise, and the fluctuation theorem 
holds also for the novel paths. We focus on a fully irreversible three-state loop, as a canonical model 
of microscopic irreversibility, finding its entropy distribution, rate of entropy production, and large 
deviation function in closed analytical form, and showing that the widely observed kink in the large 
deviation function arises solely from microscopic irreversibility. 
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Introduction. — Entropy production is a hallmark of 
nonequilibrium steady states. While entropy production 
is a system-dependent quantity, there emerge some re- 
markable universal properties: for example, the probabil- 
ity distribution of the total entropy production satisfies 
a detailed fluctuation theorem in large classes of systems 
(see, e.g., [IHli); ^^'^ ^ kink appears in its large deviation 
function (and in that of related currents) at zero entropy 
production [^-[13 • Initially, this kink has been attributed 
to specific properties of the systems under investigation, 
but a recent study indicates that it is a generic feature, 
related to the detailed fluctuation theorem [l3| . 

Most published work on fluctuation theorems and the 
related large deviation functions deals with systems that 
are reversible at the microscopic level: all transitions be- 
tween states are bi-directional. However, sheared granu- 
lar systems and chemical reactions where the products 
are cleared rapidly, are two of many important cases 
where microscopic reversibility is broken. Few recent 
publications discuss fluctuation theorems for this type 
of systems. Ohkubo has proposed a fluctuation theorem 
based on posterior probabilities [3], and Chong et al. 
showed that an integral fluctuation theorem can be de- 
rived without microscopic time reversibility [THj. 

Our aim in this letter is two- fold. First, we propose the 
study of entropy production along trajectories sampled 
at regular (small) intervals, instead of the usual sam- 
pling on each change of the system's state. This novel 
sampling is equivalent to the traditional technique, in 
the limit of vanishingly small intervals, and yields anal- 
ogous results, including the fluctuation theorems. The 
advantage is that it enables direct analysis of systems 
with microscopic irreversibility, and is more easily imple- 
mented in experiments and numerical studies. Second, 
we study the consequences of microscopic irreversibility 
by focusing on the smallest, canonical example: a fully 



irreversible three-state loop. We thus find universal fea- 
tures of the entropy production and related quantities, 
and demonstrate that the widely observed kink in the 
large deviation function at zero entropy is a certain fea- 
ture of irreversibility. 

Entropy production and two kinds of sampling. — Con- 
sider a stochastic dynamical process in a system with 
a discrete set of states. A, B,C, . . . , and with transition 
rates k{X, Y) (from state X toY). We denote the steady- 
state probability of being in state X by p{X), and con- 
sider only systems with p{X) > 0, for all states X. 

Event sampling: Imagine the system starting from 
state Xq (at the steady-state), and progressing through 
the sequence Xi,X2, ■ . .Xm- No other states occur be- 
tween Xi and Xij^i. The average time elapsed between 
two consecutive events is — \/k{Xi,Xi^i). This is the 
kind of trajectory, or path, employed in previous work on 



the subject (see, e.g., (i,[lo|,[l3lfl6t). 

Interval sampling: We sample the system at M regular 
intervals, r, 2t, . . . , Mr, and record its state at each sam- 
pling, thus defining a trajectory Xq,Xi, . . . ,Xm- The 
time gap between consecutive points on the trajectory 
is constant, = r. The system can be found in the 
same state on consecutive samplings, and it could also 
visit any number of states in between Xi and Xi+i [13]. 
One should note that this interval-sampling is readily ac- 
cessible in experiments, where one usually cannot record 
every transition between states, as would be needed for 
event-sampling. 

The total entropy production, in the steady state, is 
given by [3| 
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for either kind of trajectory. For interval sampling, 
Lu{X, Y) denotes the probability for finding the system 
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in state Y, after time r, having started at state X (at 
time zero). For event sampling, uj(X,Y) is replaced by 
k{X,Y). 

If the sampling rate is large enough, 1/t ^ 
Taa:Kx,Y k{X,Y), the most likely outcome for consecu- 
tive samplings is Xi = ^i+i, and on the rare occa- 
sions that Xi Xi+i no other states are visited in be- 
tween. Repeated visits to the same state do not con- 
tribute to the entropy ([T]), so as r — > interval sam- 
pling becomes equivalent to event sampling. Moreover, 
many of the properties found with the usual event sam- 
pling are reproduced by interval sampling, even for finite 
T. For example, the detailed fluctuation theorem 
P{stot)/ P{-stot) = exp(-stot), holds for both types of 
paths. A major advantage of interval sampling is that 
it lets us discuss situations of microscopic irreversibility: 
X — >■ F, but Y X , and we focus on this idea. 

The 3-state loop. — For the sake of clarity, and for a 
chance at a full analytical solution, we wish to study the 
simplest nonequilibrium system (with microscopic irre- 
versibility). A two-state system with non-trivial steady 
state (i.e., p{A)^p{B) > 0) is, per force, an equilibrium 
system. Thus, we are led to consider the 3-state system: 
A B, B ^ C, C —?' A, where we assume that all the 
rates are equal to 1, thus defining our unit of time. We 
later argue that despite its simplicity, this can be viewed 
as a canonical model for irreversibility. 

Using the rate equations for the system, one finds, 

0.0=3 + 36 ^/ COS -r , 
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where = uj{A,A) = lo{B,B) = u}{C,C) denotes 
the neutral transitions, w+ = uj{A,B) = uj{B,C) ~ 
uj{C, A) are the forward transitions, and w_ = uj{A, C) = 
w(i?, A) = uj{C,B) the reverse transitions. Although 
these exact expressions can be employed in the subse- 
quent calculations, we are interested in the limit r — > 
0, and in effect we use their lower-order expansions: 
ujo = 1 - T + + . .., UJ+ = T - t'^ + . .. , and 
uj_ — t"^ /2 ~ /2 + . . . . We have verified carefully that 
the final results are not affected. Note that the ratio 
w_/a;+ « r/2, for the "forbidden" reverse direction, van- 
ishes as T — > 0. 

Probability distribution of entropy production. — Since 
p{A) = p{B) = p{C) — 1/3, the first term on the rhs 
of ^ does not contribute to stot- The remainder, which 
we denote simply by s, is the entropy produced in the 
thermal bath coupled to our system. Of the three types of 



terms that appear inside the product describing s, loq/ujo, 
a;+/w_, and u!-/u!+, only the last two contribute to s, in 
equal and opposite amounts. Thus, s assumes a discrete 
spectrum of values: = mds, with ds = ln(a;+/aj_) 
and m = 0, ±1, ±2, . . . , ±M, where m = iV+ - iV_ is 
the excess number of forward (N^) over reverse {N^) 
transitions. 

The probability Pm of obtaining = to ds, is the sum 
of the weights of all the trajectories consistent with that 
value. The weight of a trajectory with iV_|_- forward, iV_- 
reverse, and A''o-neutral transitions is dj^^uj^^u}^^ . All 
values of N^,N^,Nq must be counted, subject to the 
constraints N+ - = m and N+ + N_+No^ M. For 
any finite M , one can work out explicit (cumbersome) 
expressions. Alternatively, the sums can be easily worked 
out numerically, see Fig. [TJ 
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FIG. 1: The probability pm as a function of m, the total 
number of forward and reverse transitions, for the 3-state loop 
where paths are sampled over M intervals of length r = 0.001. 
In order to make the different curves obtained for different val- 
ues of M more easily distinguishable, we plot — \n{pm) / {Mt) 
where Mr is the total length of the path. One should note 
the emergence of a kink for increasing values of M . 

The problem can be approached more elegantly using 
the generating function p(z) — X^mP™-^™- ^^^^ case, 
the generating function is clearly 

p{z) = ( Wo + H 

V z 

since its trinomial expansion yields all the possible combi- 
nations subject to the constraint N^+N^+N^) — M, and 
the z^-terms are precisely those where 7V+ — iV_ = m. 

Upon making the substitution z = e~'^'^^ = {uj-/uj+)'^, 
the generating function assumes its usual interpretation: 



p(e-^''^) = (exp(-H) = (^o + 



M 



(3) 



The time evolution of this generating function is de- 
scribed by a linear operator whose lowest eigenvalue, 
i^(/^), allows one to compute quantities of interest 0, S 
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|l3|. For now, we ignore the linear operator itself, since 
we can obtain v directly, from 



lim 

T-i-oo 



ln(exp(— /is)) 



where T = Mr is the total time length of each trajectory. 
We thus obtain 



(4) 



Interestingly, in our case this limit is achieved for any 
value of M. This helps us discuss T — ^ cx) [M oo), 
even as r — 0, for we can take the two limits indepen- 
dently. The fact that i^(/i) = — u) is a manifestation 
of the detailed fluctuation theorem . The eigenvalue 
iy{fj.) is plotted in Fig. [21 for the case of r = 0.001. Using 
the low-order approximations for the cj's we get 

KA^)«l-(r/2)^-(r/2)l-^ 

which compares very nicely with Q when r — >■ 0. 




The dominant contribution comes from an irreversible 
transition, P — )• Q and Q -/^ P, since oj{Q, P) — )■ 0, as 
T — > 0, in that case. It is in this sense that our model is 
canonical, for it suffices to focus on the effect of a single 
(dominant) irreversible transition, and ours is the small- 
est model that accomplishes that. 

The fluctuation function, x(cr), of the scaled entropy, 
a = s/ {s)T, is derived from an extremum of the Legendre 
transform of v: 



(8) 



It is possible to obtain a full analytic derivation of x{^) 
for our simple model, but this results in cumbersome ex- 
pressions. Instead, we illustrate the technique for the 
limit of small r. The two derivations yield virtually in- 
distinguishable curves, for r < 0.001, while more insight 
is gained from the simpler approximation. 

We begin by rewriting (the approximate) v{pL) as 

v{^x)^l-x-^-x-^■ X=(T/2)^ 

and find /i* that maximizes v{^) — {s)afi, using the ap- 
proximate limit (s) = ln(2/r); 



a + %/cr2 + 2t 



Inx* 



(The other root of the quadratic equation for x yields 
unphysical, complex values.) Finally, putting x = x^ 
and /i = /I* in ^{fJ.) — {s)afi^ we obtain 



x(a) = l-V^^ + aln( " + ^f + ^1 . (9) 



FIG. 2: The eigenvalue plotted as a function of /i, for 

r — 0.001. The top of the curve becomes flatter as r — >■ 0. 

The mean entropy production rate can now be derived 
from i^ip): 

(s) = dv/d^i\f^=o = T~^{uj+ - a;_)ln(tj+/w_) w ln(2/r) , 

(6) 

where we have used ujq + + a;_ = 1, and the last 
expression is the dominant behavior as r — > 0. The fact 
that the approximate limit is the same as the entropy 
produced in a single forward transition is in agreement 
with the notion that backward steps are exceedingly rare 
as T — >• and do not contribute to the average. 

The mean entropy production may also be computed 
from 



(i) =r-i^p(X)^(X,y)ln 



X,Y 



Lo{Y,X) 



(7) 



In general, the sum is dominated by the states P, Q yield- 
ing the fastest diverging uj{P, Q)/lu{Q, P) ratio, as r — 0. 



It is easy to check that this satisfies the symmetry rela- 
tion x(~cr) = x(o') + yet another manifestation of 
the detailed fluctuation theorem. 

The limiting form of x{'^) is universal: 



x{<y) 



1 — (T + CT In cr, as r — ; 



cr>0, (10) 



and x(o') 00 for cr < 0, as t —)• 0. The origin of 
the kink in x((t) resides in -I- 2r, Eq. ([9]), which 
tends to \a\ as r — > 0. Moreover, at the same limit, the 
logarithmic term diverges for tr < 0, but not for a > 0. 
The kink can be best explored through the derivatives of 



(11) 




Vct2 + 2^ 



as T — )■ 0. Note the existence of the limit r — > 
for x"(o') for a-ll cr 7^ 0. For finite r the magnitude 
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FIG. 3: The large deviation function (a), and its deriva- 
tive (b), as a function of a, for r = 10~^. The kink is more 
pronounced the smaller the value of r. 

of the apparent jump in x' is found to be of order 
x'(l) — 1) — >■ ln(2/T). The large deviation function 
x(cr) and its derivative are plotted in Fig.[3l 

For our simple model, we were able to find the gener- 
ating function (jSj by inspection. For other systems, in 
general, it can be expressed as a matrix product, 

(e"^") = u R*^v; Hxy = Xy , 

(12) 

where u = {p{A),p{B), . . .) and v is a column vector of 
ones. Then, for T — cx), 

zy(/i) = -T-MnA(/i) , (13) 

where A(^) is the largest eigenvalue of R. 

N -state ring. — It is easy to generalize the foregoing re- 
sults to an A^-state ring: Ai A2, A2 ^ A3, . . . , An ^ 
^1, ^ 3), where all rates are 1. The key ingredient 
arises from the fact that the forward transition probabil- 
ity (after time r), from Ak — > Ak+i, is then w t, while 
the "forbidden" transition probability, for Ak — Ak-i, 
is T^-^/{N - 1)! = 5nt. All of the results vahd for 
iV = 3 can be then extended to general TV, expressed as 
a function of Sj^. In particular, 

i.(M) = 1 - <55(r - S],-'^ , (14) 
from which follows 

(s) = ln(l/,5Ar) , (15) 
x{(t) = 1 - ^a'~ + A5n (16) 

— ^ — ) ' 

X (cr)lr^o = -In ( I. (17) 

The results > 0)|7-^o = 1 — cr + cr In cr, and 

x"(cr)|i— j-o — l/l""!: are universal. 

Event sampling. — The A^-state ring can be analyzed 
also with event sampling, only that then one must postu- 
late [l3| a small back reaction rate e for the "forbidden" 
transitions Ak ^ Ak~i. It is easy to show that 

i.(^) = 1 -f e - eA- _ gi-M ^ (18) 



for all iV > 3. Thus, the results from event sampling 
agree with those of interval sampling, in the limit of r — >■ 
0, provided that one sets 6 = 6^ = t^^"^ /{N — 1)! (for 
the iV-ring). This physical meaning of the small rate e is 
new to our work — indeed, for event sampling there is no 
coherent prescription on how to choose independent e's 
for the various irreversible transitions. 

Conclusion. — In this letter we have proposed the use 
of interval sampling, a novel technique for studying en- 
tropy production in nonequilibrium steady states. Most 
importantly, interval sampling allows direct analysis of 
systems with microscopic irreversibility, and is more eas- 
ily implemented in experiments. We then focused on 
the smallest model possessing irreversibility — the three- 
state loop — and argued that it may serve as a canonical 
example for systems with microscopic irreversibility, such 
as driven granular systems, in general. In this way, we 
were able to identify universal features of entropy pro- 
duction, including its large deviation function and the 
kink at zero entropy production, which is now seen to 
clearly arise from the irreversibility alone. 
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